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(54) Abstract Title 

Portable apparatus monitoring reaction of user to a performance 



(57) A portable wristwatch device 1 contains one or more sensors to detect an indirect user response to a 
performance (e.g. dance music at an event) and a user interface to provide a direct user response to the 
performance. A processor obtains a user reaction signal by combining the direct and indirect responses, and 
this is wirelessly transmitted to a remote apparatus. The sensors may sense pulse rate, temperature, 
perspiration etc. and the device may include a display 7 that may prompt the user for input via user input 
device(s) 8, 9. The device may be The user reaction information from the device is used in a music playing 
system (automatic DJ) along with additional information from other sensors located externally to the user (e.g. 
floor pressure sensors, video camera) in order to influence songs played, volume etc. 
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At least one drawing originally filed was informal and the print reproduced here is taken from a later filed formal copy. 
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MONITORING OF USER RESPONSE TO PERFORMANCES 

BACKGROUND OF INVENTION 
5 Field of Invention 

This invention relates to apparatus for monitoring the reaction of 
individuals and collections of individuals to a performance, and also to 
apparatus for producing performances modified in response to user 
reactions. Embodiments are particularly relevant to the choice of songs 
10 played at an event. 

Description of Related Art 

While the present invention is relevant to monitoring of various types of 
performance, it has particular relevance to reactions to and automatic 
generation of dance music. Terms relevant to dance music will therefore be 
used here - to help the reader to interpret this text, it is to be understood 
that the term "song" is intended to cover any piece of music that may or 
may not include a vocal element. It in particular covers dance music which 
is best characterised by a repetitive beat, but also covers other types of 
music including rock and pop music. An individual song, is referred to 
hereinafter as a "track", whilst a selection of individual songs played one 
after another in a (usually) continuous sequence is referred to as a "mix". 
The term "event" used herein may comprise a dance held in a nightclub or 
at a private party or function room, or perhaps an outdoor event such as a 
garden party or festival. 

The dance culture is increasing in popularity both in the United Kingdom 
and overseas. Perhaps the most important element in the success of the 
dance culture is the DJ-the person who selects the tracks, and the order in 
30 which the tracks are played in the mix. In some case the DJ's are becoming 
as famous as the recording artists themselves. Nightclubs often establish 
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their reputations on the basis of their choice of DJ, and popular DJs often 
have their own devout following of loyal supporters. 



The DJ builds his or her reputation both on their musical taste and as 
5 importantly on their ability to change the tracks they are playing in 
response to the reaction of the crowd. For instance, if a fast tempo record is 
played and the crowd react favourably the DJ may follow the track with 
another fast tempo track. If a slow tempo track is played and the DJ sees 
that a large number of members of the crowd leave the dance floor then the 
10 DJ may return to playing fast tempo tracks. Of course, this is a very 
simplistic view and the DJ needs to rely on an innate ability both to 
monitor the behaviour of the crowd in order to react to their demand but 
also to select tracks which directly influence the behaviour of the crowd. 

15 At present, if a person at an event wants to influence the DJ they must 
leave the dance floor to approach the DJ and ask if they will take requests 
for tracks. This is possible at smaller events such as a private party or 
wedding. In larger events this is impractical and the DJ will choose tracks 
by looking at the overall reaction of a crowd. Only the people nearest the 

20 . DJ or within line of sight will generally have a real influence. A person 
who wishes to dance out of sight of the DJ or listen from a part of the event 
away from the dance floor will therefore have little ability to influence the 
choice of tracks played. 

25 BRIEF SUMMARY OF INVENTION 

In accordance with a first aspect the invention provides a portable 
apparatus for monitoring the reaction of a user to a performance, the 
apparatus comprising: one or more sensors adapted to detect one or more 
sensor signals together constituting an indirect user response to the 

30 performance; a user interface adapted for the user to provide a direct user 
response to the performance; a processor adapted to provide a user reaction 
signal comprising a combination of the indirect user response and the direct 
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user response to the performance; and a transmitter, whereby the processor 
is adapted to control the transmitter to transmit the user reaction signal to 
remote apparatus. 

5 The portable apparatus of the invention therefore monitors the reaction of a 
user to the music played at an event and communicates this reaction to a 
remote apparatus so providing the user with the ability easily to 
communicate his or her reaction during the event. 

10 By portable we mean a device which can be worn or carried by the user, 
which is typically battery powered. It may be a wearable device, perhaps 
comprising a pendant which can be worn on a chain around the neck of a 
user. It is also envisaged that in a most preferred arrangement it may 
comprise a device which is attached to the wrist of a user. It may form a 

15 part of a wristwatch. 

According to a second aspect the invention provides a system for 
monitoring the response of users to a performance, comprising: two or 
more portable apparatuses for monitoring the reaction of a user to a 

20 performance, each such apparatus comprising one or more input devices 
each for obtaining a direct or indirect user response to the performance, a 
processor adapted to provide a user reaction signal from the user responses, 
and a transmitter, whereby the processor is adapted to control the 
transmitter to transmit the user reaction signal to a central monitoring 

25 apparatus; and a central monitoring apparatus having a receiver and a 
processor adapted to receive user reaction signals from the receiver and to 
combine them into an audience reaction signal. 

The apparatus is suitable for installation at an event, such as installation 
30 within a nightclub. The monitoring apparatus preferably includes a 
plurality of portable devices, perhaps tens or hundreds of devices 
depending on the size of the crowd. 
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In accordance with a third aspect the invention provides a music playing 
system comprising: a plurality of inputs to receive input signals containing 
information indicating the reaction of a user to a song; memory storing a 
5 plurality of song elements; and selection means adapted to select at least 
one song element or combination of song elements from the memory to be 
played, in which the selection means is adapted to select the song element 
from memory in response to the input signals. 

10 In accordance with a fourth aspect the invention provides a data carrier 
having encoded therein a computer program comprising a set of 
instructions adapted to program the processor of a handheld computing 
device to monitor the reaction of a user to a performance, the programmed 
processor being adapted to provide: a user interface adapted for the user to 

15 provide a direct user response to the performance; a user reaction signal 
derived from the direct user response to the performance; and a 
transmission of the user reaction signal through a communications interface 
of the handheld computing device to specified remote apparatus. 

20 A wide variety of handheld computing devices, particularly handheld 
personal computers and personal digital assistants (PDAs) are known in the 
art which may be adapted through the use of a suitable set of program 
instructions to provide a portable device that monitors users reactions. The 
HP Jornada is an example of a handheld personal computer, whereas the 

25 Palm Pilot family of devices are examples of PDAs 

BRIEF DESCRIPTION OF THE DRAWINGS 

There will now be described, by way of example only, one embodiment of 
the present invention with reference to the accompanying drawings of 
30 which: 



5 

Figure 1 is a perspective view of a portable device in accordance 
with the present invention; 



Figure 2 is a schematic illustration of the functional components of 
5 the device of Figure 1; 

Figure 3 is a schematic overview of a monitoring apparatus installed 
at an event which includes a remote device that receives signals 
from the portable device of Figure 1; and 

10 

Figure 4 is a schematic overview of an automatic DJ apparatus using 
a monitoring apparatus of the kind shown in Figure 3 together with 
additional ambient sensor devices. 

15 DETAILED DESCRIPTION OF THE INVENTION 

Embodiments will now be described of portable devices for monitoring 
audience member reactions to performances, of monitoring systems for 
receiving audience reactions both directly (from, for example, interactively 
through such portable devices) and indirectly (by monitoring of the 

20 environment, or by monitoring parameters of individual or multiple 
audience members), and of automatic performance generating apparatus 
(specifically automatic DJ apparatus) using the results of such monitoring. 
Although the embodiments here are focussed upon dance music and the role 
of the DJ, it will be appreciated that the principles described here can 

25 readily be employed in connection with other types of performance. 

The portable device 1 illustrated in Figure 1 of the accompanying drawings 
comprises a main body 2 that is generally rectangular. Two halves 3,4 of a 
strap extend from opposing sides of the main body 2. One half of the strap 
30 is provided with a buckle 5 whilst the other half includes a series of holes 
6a, 6b that pass through the strap. The buckle 5 permits the two halves of 
the strap 3,4 to be securely joined so that the device can be strapped to the 
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wrist of a user. In an alternative construction not shown in the drawings, 
the main body may be supported by a length of chain which permits it to be 
worn around the neck of a user. 

5 The main body 2 carries on its front face a display panel 7 which comprises 
a rectangular liquid crystal display panel. The panel comprises a matrix of 
liquid crystal elements which can be selectively controlled to display a 
range of alphanumeric characters on the display. It is also configured to 
display a range of predefined icons (not shown). It may be backlit for use 
10 in low levels of ambient light. 

The device 1 can be used to provide a direct audience member response to 
the music currently or recently playing. Alongside the display panel 7 the 
main body carries two input buttons 8,9 (an alternative would be for 

15 display panel 7 to be touch-sensitive, and for buttons 8,9 or similar inputs 
to be provided as virtual buttons through the display panel in known 
fashion for touch-sensitive displays). The buttons 8,9 can individually be 
operated by the user to input a response to the device 1. In use, the display 
7 prompts the user for an input and displays the function of each button. 

20 Changing the display permits the function of each button to be varied. The 
display 7 tells the user the function of each button at any moment in time. 
A shown in Figure 1, the display shows a letter "P" next to a first one of 
the buttons and a letter "N" next to the other. This indicates the user should 
press the first button 8 if they are enjoying an event (positive reaction) and 

25 the second button 9 if they dislike the event (negative reaction). The exact 
letter or words displayed for a given function will depend to a large extent 
on the target audience of users for the device. 

As well as or in an alternative to a simple button, an input device may be 
30 provided which can be set to any one of a range of positions by the user, 
each position corresponding to a different reaction of the user. For instance, 
a rotary dial or linear slider may be provided. The position of the dial or 
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slider at one extreme of its range of positions may correspond to most 
positive (very happy). As the dial or slider is moved to the other extreme of 
its range of positions it may move through slightly positive (less happy) to 
slightly negative reaction (unhappy) and at the other extreme indicate a 
5 very negative reaction (very unhappy) of the user. 

A positive reaction is indicative that the user is enjoying the song and a 
negative reaction indicates that they dislike the song. Of course, the 
reaction is relative and it may be that a user experiences a negative reaction 
10 to their favourite song simply because it is played at the wrong time at the 
event or because they are not in the mood to listen to that song. 

The display 7 also shows contextual information to the user. As shown in 
Figure 1 the device is telling the user the time in the top half of the display 
15 7 and so functions as a wristwatch. A button 10 on the side of the main 
body allows the user to adjust the settings of the device. 

The display may prompt the user to input information using the or each 
input device. This allows the function of an input device to be varied, with 

20 the display telling the user the function of the input device at any given- 
instant. For example, the display may prompt the user to press a particular 
button if they like a song, or to press it twice in succession if they want to 
hear a faster song or press a different button to hear a slower song. Later in 
an event, the display may prompt the user to press the button twice for a 

25 slower song, thus altering its use. Such change in function can be triggered 
by the DJ, for example, if the device 1 contains not only a transmitter but 
also a receiver that allows it to receive instructions as well as provide user 
reaction signals. 

30 Device 1 need not be specially constructed for use as a monitoring device. 
A particularly effective solution may be to program an existing handheld 
computing device, such as a handheld personal computer, a personal digital 
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assistant (PDA) or a cellular telephone. In such cases, the user interface 
choices will be those appopriate to the computing device concerned — for a 
PDA, for example, which will normally have a touch-sensitve screen, it 
may be advantageous to use virtual buttons of the kind described above. 

5 

A direct user response to the performance can thus be provided through the 
user interface of device 1. However, device 1 can also be used to provide 
an indirect user response by monitoring the users state of well being or 
physical behaviour. Device 1 therefore further comprises one or more 
10 sensors which each sense a parameter of the users state of well being or 
physical behaviour, the parameter being indicative of the reaction of the 
user to a song. 

The main body of the device also includes a small microphone (not shown) 
15 which receives acoustic signals through a small grille opening 11 in the 
front face of the main body 2. This microphone can therefore sense sounds 
provided by the user. This microphone can be used for direct user input 
(essentially a speech user interface for device 1) but can also be used for 
indirect user input - detecting particular sounds, or words, produced by the 
20 user or in his or her vicinity. 

Where speech is to be evaluated the device 1 should include a voice- 
recognition circuit. This may, for example, be adapted to detect one or 
more keywords spoken by the user. A negative response signal may be 

25 produced for one keyword and a positive signal for a different keyword. 
The voice recognition circuit may be trained to identify the user's voice 
and may have stored in memory examples of the or each keyword as spoken 
by the user. These may be entered into the memory in a training mode 
before the device can be used. The voice recognition circuit then looks for 

30 the presence of parts of the signal produced by the microphone which 
match the keywords stored in memory. Voice recognition is already known 
in the art and the reader is directed to the teachings of the voice recognition 
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circuit used in pribr art mobile phones as an example of the implementation 
of such a circuit. Voice recognition is of course implementable in software 
once the voice has been translated into an appropriate digital signal. 

5 A number of sensors (not visible) are also fitted to an under side of the 
main body 2 in contact with the skin of the user. Where the device 1 is a 
programmed computing device, these may, for example, communicate with 
the device 1 in accordance with its normal arrangements for use of 
peripheral devices. Specific types of sensor device and their significance 

10 are described below. 

A temperature sensor may be included - if an audience member is dancing 
enthusiastically, their temperature is likely to be raised. 

15 A perspiration sensor sensitive to the conductivity of an area or point on 
the users skin may be included. This can provide an indication of how hard 
the user is perspiring. A heavy rate of perspiration may be used to indicate 
vigorous dancing indicative of a positive response to a song. 

20 A pulse rate sensor may also be included - again, a high pulse rate is likely 
to indicate energetic dancing and a positive reaction. 

It will be appreciated that monitoring of these features can also in 
appropriate arrangements allow central determination of whether audience 

25 members are at risk of endangering their health (dangerously high 
temperatures or pulse rates). There are thus reasons to make such sensor 
results centrally available in the reaction signal provided from device 1 (to 
provide information centrally that may safeguard user health) but also not 
to make such sensor results centrally available (protecting the privacy of 

30 individual users). 
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Another sensor that may be used is an inclinometer or an accelerometer 
which is adapted to produce an output indicative of movement of the 
portable device. The movement of the device can be used to measure the 
reaction of the user. If the user is dancing vigorously the device will move 
( 5 around a lot which will indicate the user is reacting positively to a song. If 
the user is still the output may be interpreted as indicating that the user is 
reacting in a less positive way to a song.. 

The device 1 contains a microprocessor and a memory. The microprocessor 
10 receives the user reaction signals (whether from the user interface or from 
sensors) and provide a user reaction signal or signals as output. The 
microprocessor has access to an area of memory where program 
instructions and data are stored. 

15 As discussed above, the output signal may contain sufficient information 
for the value of each reaction signal to be extracted. Thus, the reading of a 
pulse sensor and the reading of a temperature sensor may be individually 
extracted from the output signal. 

20 Alternatively, the processor may be adapted to combine one or more 
reaction signals to produce a combined output signal from which the 
individual values of the reaction signals cannot be extracted. The combined 
output signal may encode a single overall reaction value, which may be 
equal to an average reaction of the user when considering the average of 

25 the reaction signals. The combined reaction value may be displayed on the 
display of the portable device if a display is provided. The user may 
indicate whether or not they agree with the displayed value. 

The amount of processing performed by the processor to produce the output 
30 signal will vary from application to application and will depend on the 
amount of post-processing that is performed by any central device to which 
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the output signal is transmitted and on the type of monitoring included in 
the device. 



If a number of sensors are provided, the processor may apply a set of 
5 logical rules to the output of each sensor in order to provide an overall 
reaction measurement that can be encoded in the output signaL 

For instance, if an inclinometer or accelerometer is provided it may 
produce an output signal indicating that the user is not moving. If no other 

10 sensors or inputs are available this may be interpreted as representing a 
negative reaction of the user (they are not dancing so probably dislike a 
song). If, on the other hand, an input device is operated by the user and 
produces an output indicating that the user likes the song then the output of 
the accelerometer may be overridden and ignored. The user may simply be 

15 tired or just standing still while listening to the song. 

The logical rules may define the way in which the reaction signals provided 
are combined, and which take preference over others. 

20 Figure 2 of the accompanying drawings is a schematic illustration of the 
circuitry and components located within the main body of the device 
illustrated in Figure 1 of the accompanying drawings. 

The main body 2 houses a printed circuit board 20 which carries tracks 
25 defining an electronic circuit. One of the tracks 21 is connected to the 
positive side of a battery 22 whilst another track 23 is connected to the 
negative side of the battery 22. This provides the power that is drawn by 
the circuitry. 



30 The buttons 8,9,10 on the main body 2 are also connected to tracks on the 
circuit board and the display panel 7 is connected to a multi-pin socket 24 
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which is soldered to one face of the board 20. The sensors (represented by 
block 27) are also connected to the processor. 

The tracks connected to the buttons 8,9,10 are connected to the input 
5 terminals of a processor 25. The processor 25 has access to a memory (not 
shown). An output terminal of the processor is connected through 
additional tracks on the circuit board 20 to a display driver device (not 
shown) which is in turn connected to the display socket 24. Another output 
from the processor is connected to a wireless radio transmitter 26. The 

10 transmitter 26 sends encoded signals from the device to a remote device 
located at the event. The processor encodes the transmitted signal with 
information obtained by the sensors and the buttons. An identification 
code to identify the originating device 1 may also be included. The 
transmitter may use any appropriate technology - radio-frequency 

15 communication is a particularly appropriate technology, and infra-red 
communication is a possible alternative. 

Different approaches may be adopted to providing information from 
audience members. The device 1 may continuously transmit a current 
20 reaction signal, or may transmit a signal at spaced time intervals. For 
example, a signal may be transmitted as a packet of encoded data at 1 
second, or perhaps 1 minute or perhaps 5-minute intervals. 

In an alternative arrangement, the portable device may only transmit a 
25 signal to the remote apparatus in response to a request from the remote 
apparatus. This helps to preserve the battery life of the portable device. It 
also allows a user to wear or carry the device at all times, signals only 
being transmitted when it is used at an event where an appropriate remote 
device is located. A further advantage of this mode of operation is that the 
30 user does not need to remember to switch the device on at the start of an 
event and off again at the end of the event. 
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The memory may also be used to store the response of the user between the 
transmission of signals to the remote device. The stored responses may be 
displayed by the user on an appropriate display provided on the portable 
device. 

5 

The output signal sent to the remote device may encode the response of the 
user in a number of ways. In one arrangement, the signal may encode a 
simple digital or analogue response value which may vary between an 
upper and a lower limit, one of the limits indicating a positive reaction and 
10 the other a negative reaction. The value may vary linearly between the 
extremes of the range to indicate any degree of reaction between very 
positive and very negative. 

A positive reaction is indicative that the audience member is enjoying the : 
15 song or mix of songs played at the event and a negative reaction indicates 
that he or she dislikes the song or mix of songs. Of course, the reaction is 
relative and it may be that an audience member (or as will be discussed 
below, a crowd) experiences a negative reaction to their favourite song~ 
simply because it is played at the wrong time at the event or because they- 
20 are not in the mood to listen to that song. The reaction may indicate the* 
member's preferences for the choice of songs subsequently to be played at 
the event. A positive reaction may also indicate that a person at the event 
likes the change from one song to the next at the event. 

25 In a further refinement it is envisaged that the user may operate an input 
device, such as button, to indicate that they want to "bookmark" a song that 
is playing, the portable device storing in memory data representing a 
feature of the song. This may comprise a short recording of the song 
obtained by a microphone built into the device, or a time-stamp which 

30 allows -the user to identify the song at a later time from a playlist of the 
event. 
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The portable device 1 is intended to be used in combination with a remote 
detection apparatus located at the event. Figure 3 of the accompanying 
drawings illustrates schematically a simple system suitable for installation 
in a night club which hosts a dance event. The remote apparatus comprises 
5 a central processing unit 30 which will usually be fixed on site and 
provided by the host of the event or the D J to help gather information from 
people at the event. Connected to the central processing unit by electrical 
cables are two remote devices 31,32 which monitor and detect signals 
transmitted by portable devices 1 within the club. In fact, the monitoring 
10 apparatus receives signals from every device present at the event. 

Each detection unit 31,32 comprises a radio frequency detector which 
detects the signal emitted by any portable devices 1 within its detection 
range. An amplifier increases the amplitude of the detected signals and 
15 passes the signals to a processor. The processor extracts the information 
encoded in each signal and produces an output signal which is passed down 
the electrical cable to the main processing unit 30. 

One of the remote devices 31 is located in a part of the nightclub 
20 containing a dance floor 33 within range of any portable devices worn by 
users on the dance floor. The other device 32 is located in a different area 
of the nightclub, such as in a "chill-out" room or eating area 34. This 
device only detects signals from portable devices worn by occupants of the 
room. 

25 

The main processing unit 30 receives signals along the two cables from 
each of the detectors 31,32. The signals provide the processor with 
information about the location of the wearer of a portable device and also 
the reaction of the wearer to the music that is played. The amount and type 
30 of data received depends on the content of the signals transmitted by the 
portable devices. It is envisaged that some devices may transmit more types 
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of information than others, i.e. some may have increased functionality 
compared with other devices. 

The device 1 may be adapted to receive, as well as to transmit, information. 
5 The device 1 may also include a detector which is adapted to receive a 
signal transmitted by a remote device. The information encoded in the 
signal may be processed by the processor. It may be used to change the 
function of any input devices on the portable device, or to display messages 
or instructions on the display (where provided). 

10 

The remote device could, for example, send a request by a DJ or club 
owner to each or an individual portable device to ask the user for a specific 
reaction to a song. For example, the user may be asked if the tempo of the 
song playing is too fast or too slow, or if the music is too loud or quite. 
15 This provides a full interaction between the DJ and a user of a portable 
device. 

A display may be provided at the event upon which information can be 
presented by the user of the portable device. A DJ could, for example, ask 
20 the user a question such as "Do you like this song?" The reaction of the 
user to the question can then be monitored. 

The portable apparatus thus monitors the reaction of the user and transmits 
a signal to a remote device which can be adapted to select the next or any 
25 other subsequent track to be played in dependence upon the users response. 
This allows the user to interact with the tracks played and influence the 
course of the event. 

The data received by the processing unit 30 is stored in a memory 35 and 
30 can subsequently be used to produce a control signal which is used either 
by a human DJ or an automatic DJ apparatus 36 to influence the next or 
subsequent tracks that are played at the event. For example, if a number of 



16 

signals are received from portable devices asking for the track to be 
changed then the track may be changed by the DJ before it would normally 
end. If a large number of requests for a faster tempo track are received then 
the music may be changed accordingly. 

5 

By choosing which song to play in response to the crowd reaction, the mix 
can automatically be tailored in order to follow the wishes of the crowd. In 
effect, control of the mix is at least partially handed over to the people in 
the crowd and it is the reaction of the crowd that determines which tracks 
10 are played. Every event may therefore be unique to the crowd's 
preferences. 

Such monitoring apparatus can thus be used as a tool to assist a DJ by 
selecting tracks from the memory but leaving the DJ to confirm whether or 
15 not to play the selected track. For instance, this gives the DJ extra feedback 
about the crowd reaction and can assist both skilled and inexperienced DJs 
alike in pleasing the crowd. However, this type of monitoring apparatus is 
especially useful in providing a quantified audience reaction to an 
automatic DJ adapted to reproduce the function of a human DJ. 

20 

The apparatus illustrated in Figure 4 shows the integration of user devices 
as shown in Figures 1 and 2 with environmental monitoring to provide a 
comprehensive monitoring system used to provide user input to an 
automatic DJ system. The system is installed in a hypothetical small 
25 nightclub 101 having a single dance floor 102 and a chill-out room/bar area 
103. The apparatus monitors the reaction of people in the club to the songs 
that are being played. It automatically selects songs to be played in 
response to the monitored reactions of the people, replacing the traditional 
human DJ at the club. 

30 

The apparatus comprises a centrally located automatic DJ 104 which is 
partially housed within a secure booth adjacent the dance floor 102 inside 
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the nightclub 101. The automatic DJ 104 comprises a selection apparatus 
105 and a monitoring apparatus (part of which is housed-fn the booth). The 
selection apparatus 105 selects the songs, which are played in response to 
crowd reaction signals supplied to the selection apparatus by the 
5 monitoring apparatus. 



The monitoring apparatus comprises a centrally located processing unit 106 
located within the secure booth. This may be provided by a microprocessor, 
which forms a part of a personal computer, or may be a dedicated 
10 processing device. The monitoring apparatus is here shown as separate 
from the automatic DJ 104, but it will be appreciated that these can be 
integrated together, and may operate under the control of a common 
processor. 



15 Connected to the processing unit 106 by electrical cables 107 are a range of 
sensing devices which are spaced around the nightclub. If preferred, a 
wireless link may be provided between each sensing device and the 
processor. 

20 Many differing types of sensing device can be employed, as there are many 
types of behaviour which may be monitored to indicate the reaction of a 
crowd. These may include, but are not limited to: 

Sensors that monitor the number of people dancing to a song, 
Sensors that monitor the number of people on a dance floor, 
25 Sensors that monitor the temperature at the event, 

Sensors that monitor sound or gestures made by people at the event, 
Sensors that directly receive answers to questions asked to people at 
the event, i.e. " is this too slow/fast, are you tired etc" 
The more sensors and the more parameters monitored the more reliable an 
30 estimate of the reaction of the crowd to the event can be produced. 
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Each sensing device produces an electrical output signal which can be used 
to estimate the reaction of people present in the club to the songs that are 
being played. These raw output signals are encoded in signals sent along 
the cables to the central processor. The processor 106 extracts the 
5 information encoded in each signal and produces one or more crowd 
reaction signals. 

In the example shown in Figure 4 several different sensing devices are 
located in a part of the nightclub containing a dance floor. Other devices 
10 are located in a different area of the nightclub, such as in a "chill-out" 
room or eating area. 

Different types of sensing device used or usable in the Figure 4 
embodiment will now be discussed. 

15 

A dance floor sensor is adapted to monitor people on a dance floor at the 
event. Such a dance floor sensor may generate an output signal dependent 
upon the presence of or the behaviour of the people on the dance floor. 
One such sensor is the array of independent pressure sensors 108 which 
20 may be located on, within or below the surface of the dance floor, covering 
the whole of or a portion of the dance floor. For example, an array of 6- 
inch square pressure sensors may be provided which cover an entire area of 
the dance floor. 

25 The pressure sensors may produce a binary output signal having a first 
value if a person stands or sits or lies on the sensor and a second value 
otherwise. Alternatively, the pressure sensors may produce an output signal 
which indicates the actual pressure applied to a sensor at any time. This 
permits more sophisticated monitoring. 



30 
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Where a number of dance floors are provided at an event, such as in a large 
nightclub, each floor or a subset of the total number of floors may be 
provided with at least one dance floor sensor. 

5 A digital video camera 109 is adapted to capture a digital image of at least 
a portion of a dance floor, and has associated with it an image processor 
adapted to process the digital image to identify the number of people 
present in the captured image and/or the behaviour of those people. This is 
preferably a thermal imaging camera but may alternatively or additionally 
10 comprise a video camera. 

At many events, in order to create the correct ambience the dance floor may 
be poorly lit. This makes it difficult for a video camera to capture a 
satisfactory image and also makes it difficult to process the image. A 
15 thermal imaging camera may be preferred in such an environment. 

The number of people on the dance floor may be estimated from the 
average temperature of an entire image or by identifying individual people 
from the features of the image. An estimate based on an average may prove 
20 reasonably accurate as the larger the number of people the greater the^ 
temperature of the crowd/room is likely to be. 

Where a video camera is provided, a laser may be provided which emits a 
pattern of light towards the dance floor captured in the video image, the 
25 image processing means monitoring the pattern of the light from the laser 
in the image to detect the presence of people in the image. 

A microphone 110 or other acoustic sensor is adapted to detect sounds 
which are produced by people at the event. The sound of the song which is 
30 being played at the time of measurement may be subtracted from the output 
signal from the or each microphone to isolate the sound made from the 
people at the event from the music. 
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This sensor may enable production of a crowd reaction signal which has a 
value dependent upon the amplitude of the sound made by the people in the 
crowd. For example, the crowd reaction signal may indicate a positive 
5 crowd reaction if the crowd are making a lot of noise and a negative 
reaction if the crowd are silent. The louder the crowd the more positive a 
crowd reaction signal may be produced. 

In a more sophisticated embodiment, the type of sound produced by the 
10 crowd may be used to determine the reaction of the crowd. The type of 
sound may be monitored by appropriate signal processing of the output of 
the or each or selected ones of the acoustic sensors/microphones. 

A crowd reaction signal indicative of a positive crowd reaction may be 
15 generated if the output of the microphone has a large high frequency 
content relative to its low frequency content. This type of sound 
corresponds to whistling and screaming. 

Similarly, a crowd reaction signal indicative of a negative reaction may be 
20 generated if the output of the microphone has a small high frequency 
content relative to its low frequency content. This corresponds to a crowd 
booing or shouting. 

A temperature sensor 111 is adapted to monitor the temperature at the 
25 event. The more people who are on a dance floor the higher the 
temperature. Similarly, the more people enjoy a track the more energetic 
their dancing will become. 

The temperature sensor may, for example, comprise an infrared sensor or a 
30 thermal imaging camera. The images captured by a thermal imaging camera 
may be analysed by an image processor to determine the body temperature 
of one or more people at an event, and especially of one or more people on 
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a dance floor. Alternatively, as suggested hereinbefore an average 
temperature for a whole or portion of a captured thermal image may be 
used. 

5 A crowd reaction signal may be produced which is indicative of a positive 
crowd reaction if the temperature is increasing or exceeds a predefined 
threshold, and indicate of a negative reaction if the temperature drops or is 
below the threshold. 

10 In a sophisticated system, the output of the temperature sensor may be 
cross-correlated with a measurement of the number of people on the dance 
floor and/or the type of song that is being played. 

A humidity sensor may be provided which is adapted to monitor the amount 
15 of moisture in the air. This may be located inside or in front of a duct 
through which an air conditioning unit extracts air from around a dance 
floor. It is expected that the humidity will be higher when lots of people 
are dancing vigorously and this can indicate a positive crowd reaction. 

20 Of course, in other embodiments a smaller range of sensors may be 
provided whilst in larger nightclubs more sensors may be desirable. A 
digital video camera 112 is also provided in the chill out room which 
captures images of the room. 

25 Of course, the presence of a large number of people on a dance floor does 
not necessarily provide a complete picture of crowd reaction. The 
behaviour of the people on the dance floor is also an indication of whether 
or not they are enjoying the event. 

30 The dance floor sensors may be adapted to monitor at least one aspect of 
the behaviour of the portion of the crowd. This may be adapted to monitor 
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how vigorously the people are dancing by monitoring a pattern of 
movement of an individual or a crowd. 

Where pressure sensors are provided, the rate of change of the output signal 
from the or each sensor may be used to provided an indication of the 
movement of the people on the dance floor. For example, if everyone is 
stood still the sensors will detect the presence of the people but the output 
of the sensors will remain reasonably constant. If everyone is jumping in 
time to a track, the output of the sensors will vary in time with the jumping 
of the crowd. 

The variation in the output of the pressure sensors over time may be 
detected and used. For example, this may be used to distinguish between 
vigorous dancing and a mass exodus of the dance floor. It is expected that 
these two different types of movement will have unique time-signature 
patterns allowing them to be distinguished. A neural network algorithm 
that is trained to distinguish positive reaction movement patterns from 
negative patterns of movement may be used in association with main 
processor 106. 

The main processor 106 also receives signals from a number of remote 
devices which are worn or carried by people 112 in the nightclub. These 
devices, which may be of the type shown in Figures 1 and 2, monitor the 
users reaction and transmit information wirelessly to receivers 113 located 
next to the dance floor or a receiver in the chill out room. The receiver 
sends the received signals back to the processor 106. 

The data received by the processor 106 is stored in an area of electrical 
memory connected to the processor. 

The processor produces one or more crowd reaction signals from the 
information contained in the output signals produced by each of the sensing 
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devices. For the sensors provided in the example, the processor initially 
generates a crowd reaction signal for each received signal. 

Simple examples of logical rules for producing reaction signals that can be 
5 applied are as follows: 

If the pressure sensors and/or the camera indicate a large number of people 
are present on the dance floor then the reaction to the currently playing 
song can be considered positive-they like the song; 

10 

If the pressure sensors and/or the camera indicate that a large percentage of 
the people on the dance floor are dancing then the reaction may be 
considered to be positive; 

15 If the sensors indicate that the ratio of people on the dance floor to people 
in the chill-out room is low then the reaction may be considered negative; 

If the sensors indicate an increase in the number of people on the dance 
floor over time then the reaction may be considered positive; 

20 

If the sensors indicate a decrease in the number of people on the dance 
floor over time the reaction may be considered to be negative; 

A positive reaction may be generated if the number of people on the dance 
25 floor exceeds a predetermined threshold. Conversely, if the number of 
people falls below the threshold the crowd reaction signal may indicate a 
negative reaction. The value of the threshold may be varied as a function 
of a parameter of the event. For instance, if the event is a dance at a 
nightclub a different threshold may be used for different nights of the 
30 week. For regular events, the value of the threshold may be adjusted by 
monitoring the number of people on the dance floor over a number of 
events and calculating an average value, the threshold corresponding to the 



24 

calculated average value. The threshold may also be adjusted as a function 
of the total number of people at the event. 

If the temperature on the dance floor is increasing this may be used to 
indicate a positive reaction (more heat equals more people or more 
vigorous dancing). 

The value of the crowd reaction signal may be scaled by a scaling factor to 
take into account parameters such as the time of day, day of the week, or 
the time since the start of the event. For instance, a greater percentage 
should be on the dance floor at the middle of an event than at the start of 
the event. Other context information such as the time of day/night, day of 
the week etc. can be employed. 

Having produced a number of crowd reaction signals, the processor then 
applies a number of logical rules to the individual crowd reaction signals to 
produce an overall crowd reaction signal. For example:- 

If the number of people on the dance floor is high (indicated by the 
pressure sensors) but they are not moving much (indicated by a low 
temperature) the reaction is considered slightly positive. If they are moving 
a lot then a very positive overall reaction signal is generated. 

If the number of people on the dance floor is high and they are moving a lot 
but the crowd reaction signal produced from the output of a microphone is 
negative (perhaps because the crowd are booing or hissing) then the 
microphone reaction signal will dominate and a negative overall reaction 
signal value will be produced. 

Having produced an overall crowd reaction signal the processor passes the 
signal to the selection means. The selection means (most obviously 
implemented as a processor operating under appropriate program control) is 
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adapted to select a song to play in the nightclub from a large collection of 
tracks stored in the memory. An index list is also stored in the memory 
which comprises at least one classification value for each stored track. For 
instance, the index for a track may indicate the tempo of the track, how the 
5 tempo changes during the duration of the track etc. 

The selection means may generate a selection value from the one or more 
control signals and select a track to play from the memory by comparing 
the selection value with the index for a stored track. For example, the 
10 selection value may comprise a tempo value, and a track having an index 
representing that tempo may be selected. If a number of tracks are stored in 
memory which match the criteria of the selection value then the track may 
be selected at random from all the possible matches. 

15 The selection means processes the crowd reaction signal together with 
historical data corresponding to the current playing song and previously 
played songs. In this simple example, the historical data indicates the 
tempo of the current playing song and the tempo of the song prior to it. 
Logical rules are then applied to the historical data and the crowd reaction 

20 signal value to determine the tempo of the next song that should be played; 

The logical rules may include the following: 

If, when the automatic DJ increases the tempo of the songs by cross-fading 
25 to a new song faster than the previous song a positive crowd reaction is 
identified then the automatic DJ should ensure that the tempo continues to 
increase by selection of a faster tempo track or stays the same; 

If, when the automatic DJ increases the tempo of the songs by cross fading 
30 to a new song faster than the previous song a negative crowd reaction is 
identified then the automatic DJ should ensure that the tempo decreases by 
selection of a slower tempo track; 
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If, when the automatic DJ deceases the tempo of the songs by cross-fading 
to a new song slower than the previous song a positive crowd reaction is 
identified then the automatic DJ should ensure that the tempo continues to 
decease by selection of a slower tempo track or stays the same; 

If, when the automatic DJ decreases the tempo of the songs by cross-fading 
to a new song slower than the previous song a negative crowd reaction is 
identified then the automatic DJ should ensure that the tempo is increased 
by selection of a faster tempo track. 

Having determined the tempo of the next song to play the selection means 
chooses a song from the memory using the index list to ensure it has the 
required tempo. If more than one track meets the criteria a track may be 
picked at random from all the matches. 

The selection means also writes to the memory a set of reaction marker 
values, each value being associated with a played track and indicating 
whether or not it generated a positive or a negative crowd reaction. When 
subsequently choosing between tracks that meet the tempo criteria, the 
track with the most positive reaction marker value associated with it may 
. be selected. 

The marker values may be updated each time a track is played and the 
reaction of the crowd is monitored. For example, the marker value may 
comprise an average reaction value determined once the track has been 
played a number of times at an event. 

The apparatus may be adapted to delete a track from the memory if the 
marker value associated with the track is indicative of a very unpopular 
track. A marker may indicate such a low popularity if it corresponds to a 
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track that has been played a predefined number of times and has a 
popularity below a predefined threshold. 

The selection apparatus may also be arranged to control other aspects of the 
5 event such as the volume of the songs that are played and the pattern and 
intensity of lighting used at the event, smoke machines or foam/ bubble 
generators etc. 

The discussion aboves relates essentially to musical performance, but it 
10 will be appreciated that other forms of performance can be modified in the 
same way. For example, visual art can be created according to rules and 
modified according to user reaction to the current form of the art - this 
could apply to static or video images. 

15 In a modification (again most obviously applicable to music but potentially 
applicable to other forms of performance), the remote device may not affect 
the tracks being played or to be played but instead store the users reactions 
during at least a portion of the event together with information about the 
tracks played in that portion of the event. The remote device may select the 

20 tracks which generated the most positive reactions from the user, and copy 
the selected tracks to a storage medium. This medium may be a web page 
or a compact disc or minidisc. The site can then be viewed by anyone after 
the event, or the CD may be sold to the crowd at the end of an event. 
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CLAIMS 



1. A portable apparatus for monitoring the reaction of a user to a 
performance, the apparatus comprising: 
5 one or more sensors adapted to detect one or more sensor signals together 
constituting an indirect user response to the performance; 
a user interface adapted for the user to provide a direct user response to the 
performance; 

a processor adapted to provide a user reaction signal comprising a 
10 combination of the indirect user response and the direct user response to 
the performance; and 

a transmitter, whereby the processor is adapted to control the transmitter to 
transmit the user reaction signal to remote apparatus. 

15 2. Apparatus according to claim 1 in which the processor combines one 
or more of the sensor signals and the direct user response to produce an 
user reaction signal from which the individual input values cannot be 
extracted. 

20 3. Apparatus according to claim 1 in which the processor applies a set 
of logical rules to the sensor signals and the direct user response in order to 
provide an overall reaction measurement as the user reaction signal. 

4. Apparatus according to claim 3 in which the logical rules define 
25 which of the separate sensor signals and the direct user response take 

preference over others of the separate sensor signals and the direct user 
response. 

5. Apparatus according to claim 1 wherein the user interface includes a 
30 display adapted to display under the control of the processor prompts to the 

user to input information using the user interface. 
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6. Apparatus according to claim 1 in which the one or more sensors 
which each sense a parameter of the user's state of well being or physical 
behaviour indicative of the user's response to the performance. 

5 7. Apparatus according to claim 6 in which at least one of the sensors 
is a biometric sensor. 

8. Apparatus according to claim 6 in which at least one of the sensors 
is a microphone. 

10 

9. Apparatus according to claim 6 in which at least one of the sensors 
is an inclinometer. 

10. Apparatus according to claim 1 in which the user reaction signal 
15 output by the transmitter uniquely identifies the portable apparatus. 

11. Apparatus according to claim 1 in which the transmitter is a radio- 
frequency transmitter adapted to transmit an encoded radio-frequency 
signal. 

20 

12. Apparatus according to claim 1, wherein the apparatus further 
comprises a receiver. 

13. Apparatus according to claim 12, in which the apparatus is adapted 
25 only to transmit the user reaction signal in response to a request received 

through the receiver. 

14. A system for monitoring the response of users to a performance, 
comprising: 

30 two or more portable apparatuses for monitoring the reaction of a 

user to a performance, each such apparatus comprising one or more input 
devices each for obtaining a direct or indirect user response to the 
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performance, a processor adapted to provide a user reaction signal from the 
user responses, and a transmitter, whereby the processor is adapted to 
control the transmitter to transmit the user reaction signal to a central 
monitoring apparatus; and 

a central monitoring apparatus having a receiver and a processor 
adapted to receive user reaction signals from the receiver and to combine 
them into an audience reaction signal. 

15. A performance generating system, comprising: 

two or more portable apparatuses for monitoring the reaction 
of a user to a performance, each such apparatus comprising one or more 
input devices each for obtaining a direct or indirect user response to the 
performance, a processor adapted to provide a user reaction signal from the 
direct or indirect user responses, and a transmitter, whereby the processor 
is adapted to control the transmitter to transmit the user reaction signal to a 
central performance generator; and 

a central monitoring apparatus having a receiver, a memory 
containing a plurality of performance elements, and a processor adapted to 
receive user reaction signals from the receiver and to generate a 
performance comprising at least one performance element selected from the 
memory in response to the user reaction signals. 

16. A performance generating system as claimed in claim 15 wherein the 
performance comprises a musical performance, and wherein the 
performance elements comprise song elements. 

1 7. A music playing system comprising: 

a plurality of inputs to receive input signals containing 
information indicating the reaction of a user to a song: 
memory storing a plurality of song elements; and 
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selection means adapted to select at least one song element or combination 
of song elements from the memory to be played, in which the selection 
means is adapted to select the song element from memory in response to 
the input signals. 

18. A data carrier having encoded therein a computer program 
comprising a set of instructions adapted to program the processor of a 
handheld computing device to monitor the reaction of a user to a 
performance, the programmed processor being adapted to provide: 

a user interface adapted for the user to provide a direct user response 
to the performance; 

a user reaction signal derived from the direct user response to the 
performance; and 

a transmission of the user reaction signal through a communications 
interface of the handheld computing device to specified remote apparatus. 
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